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ABSTRACT 

The purpose of this study was to examine the effects of information and communication technologies (ICT) on 
students’ math and science achievement. Recently, ICT has been widely used in classrooms for teaching and 
learning purposes. Therefore, it is important to investigate how these technological developments affect students’ 
performance at school. The data for this study comes from the 2009 administration of The Programme for 
International Student Assessment (PISA), an internationally standardized assessment administered to 15-year-old 
students (9 th grades) in schools. The sample includes 4996 students in Turkey. Hierarchical linear modeling was 
used for analyzing the effects of ICT in student and school levels by using ICT-related variables such as 
technology scores and ICT availability at home, etc. The results indicated that students’ familiarity with ICT and 
their exposure to technology helped to explain math and science achievement gaps between individuals and 
schools. ICT is an important factor that should be taken into consideration when designing classroom 
environments. 
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INTRODUCTION 

In recent years, computers have been used extensively for various reasons by wide user groups. School-age 
children use computers for entertainment, communication, and education, etc. Over the past few years, due to 
improvements in technology, computers and related technologies have become cheaper and more sophisticated. 
That is why households are both able and willing to buy computers for their children. They hope to give them 
the chance to become advanced computer users. Lauman (2000) stated that “not only is the number of computers 
in education growing exponentially, but also the number of computers in the home is growing at a rapid rate” (p. 
196). Despite the increase in the number of computers and related technologies, everyone does not have the 
same access to these technologies: “Media availability varies depending on such things as child’s age, gender, 
race/ethnicity, family socioeconomic status, and so forth” (Roberts et al., 1999, p.9). The economic level of the 
countries might also affect the availability of media for school-age children either at school or at home. 

Parents believe that using computers may increase their children’s academic achievement and future job 
opportunities (Ortiz et. al, 2011); therefore they buy computers with an internet connection to help their children 
succeed in school (Turow, 1999). Today’s computer revolution provides cheaper and better home computers that 
allow students to practice what they have learned at school (Stock and Fishman, 2010). Although there is an 
agreement among researchers that computers are useful for learning opportunities, Becker (2000) found that 
students are more likely to use home computers for entertainment than for school related purposes. There are 
countless things that can be done with computer applications, and some of these applications might have latent 
impacts on children’s development. For instance, computer games might be considered a waste of time by some 
parents. However, they may have positive effects on children’s cognitive development (Hamlen, 2011; Li and 
Atkins, 2004). By spending time with the computers, children can learn how to “read and utilize the information 
on computer screens” (Subrahmanyam et al., 2001, p. 14). Using computers can also improve children’s visual 
attention because some applications require users to keep track of or control many activities at the same time. 
Durkin and Barber (2002) also found that computer games have positive impacts on adolescents. 

Children are not only exposed to technology at home but also at school by new information and communications 
technologies (ICT). Due to having new computers and related technologies, schools are in need of new 
technology plans and designs. According to Kozma (2003), “Teachers in many countries are beginning to use 
ICT to help change classroom teaching and learning, and are integrating technology into the curriculum.” (p. 
13). “Therefore, it is necessary to develop strategies for students to effectively use computers and advanced 
communication technologies that can help them to improve their academic performance.” (Lee et al., 2009, p. 
226). According to analyses of U.S. data (NCES, 2001), teachers’ computer use for certain activities at school 
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positively affects students’ science achievement. Papanastasiou et al. (2003) argued that students who have 
available computers at home and in the library have higher levels of science literacy. Lee et al. (2009) found in 
their study that students who were using computer 1 hour per day had better math scores. Kim and Chang (2010) 
stated that computer use for math was associated with reducing the achievement gap among different diverse 
backgrounds. It is obvious that there might be many factors affecting students’ science and math performance. 
Technology is one of these factors; that is why it is important to explore how we can explain students’ science 
and math achievements by looking at their use and accessibility of computers and related technologies, as 
suggested by Subrahmanyam et al. (2001). Notten and Kraaykamp (2009) stated that science performance is 
positively affected if there is a positive reading climate and computer availability at home. They also mentioned 
that “the absence of a television set at home seems to narrow a child’s worldview and knowledge of science.” (p. 
379). According to Attewell and Battle (1999), mathematical performance was positively associated with having 
a home computer. Dumais (2009) also mentioned that using computers for fun was related to increasing math 
achievement. 

The aim of this study was to investigate how using computers and related technologies affect science and math 
performance among students. 

METHOD 

The data for this study come from the 2009 assessment of The Programme for International Student Assessment 
(PISA) that is an internationally standardized assessment jointly developed by participating economies and 
administered to 15-year-olds (9 th graders) in schools. PISA assesses the domains of reading, mathematical and 
scientific literacy that is covered not merely in terms of mastery of the school curriculum, but in terms of 
important knowledge and skills needed in real life. Besides assessing these specified domains, PISA includes 
student, parent and school surveys to gather information on various social, cultural and economic factors such as 
students’ and parents’ background, and their attitudes towards ICT. The sample includes 4996 students 
(male=2551, female=2445) from 170 schools in Turkey. One hundred sixty nine of 170 schools were public 
schools while there was only one privately funded school in the sample. Student level variables were obtained 
from the PISA 2009 student and ICT survey, and school-related variables were obtained from the PISA 2009 
school survey. 

Obtaining Technology Scores 

To quantify students’ exposure to technology, the questions in the PISA Student & ICT Survey were used. The 
survey includes questions about several topics such as students’ possession of technological devices and how 
frequently they use these devices at school and home. 

The technology scores from the ICT survey were obtained using the Graded Response Model that is a 
polytomous item response theory (IRT) model developed by Samejima (1969) for analyzing cognitive processes. 
The model is similar to the Bimbaum’s (1968) two-parameter IRT model in terms of dichotomization process. In 
Graded Response Model, the response categories (k) are dichotomized into two categories: (1) greater or equal 
to score category k; (2) less than score category k. With k response categories, there are k - 1 or j boundaries 
between the categories. For each between-category boundary, an operating-characteristic curve is estimated. 
These curves can be found by using the following equation: 

p v = t+ 

where PJ is the probability of selecting category j or higher, aj is the item discrimination for item i, 0 is the latent 

trait, by is the category-boundary parameter (threshold) for category j in item i. For k response categories, k-1 (or 
j) category-boundary parameters (by) are estimated. These parameters basically represent the ability level 
necessary to have a 50% chance of responding in a category above the j th between-category boundary. In the 
present study, the ICT survey items have either four or five response categories that provide three and four 
between-category boundaries, respectively. Using the given formula, a technology score representing students’ 
familiarity and confidence with ICT is estimated for each student. 

Hierarchical Data Analysis 

In this study, hierarchical linear modeling (HLM) was used for analyzing the effects of technology on students’ 
achievement. HLM focuses on the effects of social variables on behavior or performance. It allows examining 
the variance in hierarchical data structures where students are nested within classes and schools. The relative 
variation in the outcome measures, between students within the same school and between schools can therefore 
be evaluated. 
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For hierarchical linear modeling, lme4 package (Bates, Maechler & Bolker, 2007) in R was used. Before 
conducting HLM analyses, several assumptions were addressed to determine the adequacy of the hierarchical 
modeling. To see if student-level residuals are normally distributed, a histogram of observed residuals was 
generated. If the distribution resembles a normal distribution, it can be concluded that the level-1 errors are 
normally distributed (Raudenbush & Bryk, 2002). Second, multivariate normality of the school-level residuals 
was checked by examining the Q-Q plot of expected and observed Mahalonobis distance. A 45 degree line is the 
evidence of the multivariate normality of the level-2 residuals. Also, homogeneity of level 1 variance was 
checked. There were four hierarchical models fitted by using math and science scores as an independent variable 
and independent variables such as technology scores (TECH), socioeconomic status (SES), ICT use at home 
(ICTHOME), confidence in using computers (HIGHCONF), school size (SCHSIZE) and ratio of computers at 
school and school size (RATCOMP). The same models were fitted for both math and science scores. Table 1 
gives a summary of the HLM models used for the data analysis. 

Table 1: Hierarchical linear models used for data analysis 


One-way random effects ANOVA 


V u = Poj +■ 

(Level 1 - Students) 

= /« + “9j 

(Level 2 - Schools) 


Random intercept model: Model 1 

Yij = p ej + S lJ (TECH") +£ tJ (SES) + ijj 


Pe } = + “a; 

Pi i = Tio 

fej = Yw 


Random intercept model: Model 2 

Yq = p 0 j + 8 1 j{TECH) +fe f (SES} + tf v ([CT HOME) + (HIGHCONF) + r tj 

fly — 


Rj’ := Fio 


Atj = La® 

Aj = >a® 

Aj — Yao 

Random intercept model: Model 3 

Yij = Aj + EAjCSFJ:) + (ICTHOME) + ^(HIGHCONF) + 

ftj = + r* x (SCH5iZE) + y Dl ( RATCOMP ) + 

ftj ~ El® 

Aj = ft® 

Aj = fto 

Aj = 

Note: Yy is students’ math or science score in the 2009 administration of PISA. 
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RESULTS 

First, HLM model assumptions were checked. A histogram of observed residuals was generated. The 
distribution was fairly normal (see Figure 1). Multivariate normality of the school-level residuals was checked 
by examining the Q-Q plot of expected and observed Mahalonobis distance. The plot had a 45 degree line 
between two variables for both math and science (see Figure 2). That was the evidence of the multivariate 
normality of the level-2 residuals. Lastly, homogeneity of level 1 variance was checked by using chi-square test. 
The test result showed that the hypothesis of homogenous variance was failed to reject (p > 0 .05). 




Figure 1. Distributions of student-level residuals of math (left) and science (right) scores 




Figure 2. Q-Q plots of observed and expected school-level residuals in math (left) and science (right) scores 


After checking model assumptions, the HLM analyses were performed. Table 2 shows the results of the HLM 
analyses. As mentioned earlier, the first model is the one-way random effects model that accounts for variance 
between individuals and schools without any covariate. This model was used as a baseline for comparison with 
other three models that include several covariates in level 1 (student) and level 2 (school). In each step, 
technology-related variables were included in the model. Intraclass correlations (ICC) were calculated in model 
1 for both math and science scores by finding the ratio of level 2 variance to the total variance (i.e. Level 2 
variance / Level 1 variance + Level 2 variance). The ICCs were .62 and .67 for science and math scores, 
respectively. These ICCs showed that 62% variability in science scores and 67% variability in math scores can 
be explained by the variability between schools. These results indicated that there was a huge achievement 
difference between the schools in Turkey sample of PISA. The next models were used to explain these 
achievement gaps between schools by adding ICT-related variables to the models. 


Technology scores obtained from the ICT survey was not a strong predictor of science and math scores by itself. 
However, when it was used with other ICT-related variables, it was a significant predictor in all three models. 
The availability of ICT at home (ICTHOME) and confidence in using computers (HIGHCONF) were other 
important predictors of math and science performance in addition to the technology scores. Model 3 included 
two additional variables in school level: school size (SCHSIZE) and ratio of computers at school and school size 
(RATCOMP). Both variables were not statistically significant. 
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Table 2: A summary of fixed and random effect estimates from four HLM models 


Fixed Effect 

One-way Random 
Effects 

Model 1 

Model 2 

Model 3 


Coefficient 

SE 

Coefficient 

SE 

Coefficient 

SE 

Coefficient 

SE 


444.909 

4.921 455.4318 (S) 

4.5838 

457.0184 

4.5646 - 

490.3880 (S) 

10.9657 

Intercept (y 0 o) 

(S) 

435.047 

5.923 449.7068 (M) 

5.4447 

(S) 

451.0112 

5.4386 489.1635 (M) 

13.1730 


(M) 




(M) 





- 

- 

3.5069 (S) 

1.0549 

8.7179 (S) 

1.4145 : 

8.635213 (S) 

1.4148 

TECH (y,o) 

- 

- 

3.4073 

1.1307 

8.7283 

1.5144 

8.6464 (M) 

1.5147 




(M) 


(M) 





- 

- 

9.2202 (S) 

0.7782 

7.5709 (S) 

0.8545 

7.5818 (S) 

0.8544 

SES (y 20 ) 

- 

12.4057 (M) 

0.8348 

10.5984 

0.9152 

10.6041 (M) 

0.9151 






(M) 





- 

- 

- 

- 

2.6854 (S) 

0.9251 

2.6912 (S) 

0.9251 

ICTHOME (y 30 ) 

- 

- 

- 

- 

2.6793 

0.9905 

2.6794 (M) 

0.9904 






(M) 




HIGHCONF 

- 

- 

- 

- 

5.6896 (S) 

0.7128 

5.6766 (S) 

0.7128 

(Y4o) 

- 

- 

- 

- 

6.2660 (M) 

0.7630 

6.2532 (M) 

0.7629 


- 

- 

- 

- 

- 

- 

-0.0208 

0.0070 

SCHSIZE (yoi) 

- 

- 

- 

- 

- 

- 

(S)* 

0.0235 

0.0084 








(M)* 


RATCOMP 

(Y 02 ) 

- 

- 

- 

- 

- 

- 

-63.081 

25.1399 

“ 

- 

- 

- 

- 

- 

(S)*- 

73.827(M)* 

30.1966 

Fit Statistics 

One-way 

Random 


Model 1 

Model 2 


Model 3 



Effects 







Variance estimates 









Level 1 variance 

2415.8 (S) 


2353.6 (S) 

2280.3 (S) 


2280.8 (S) 

(fOj) 

Level 2 variance 

2822.8 (M) 

3986.8 (S) 


2701.7 (M) 

3308.4 (S) 

2611.0 (M) 
3221.7 (S) 

2611.0 (M) 
3000.2 (S) 

(u 0 j) 


5811.5 (M) 


4735.5 (M) 

4658.1 (M) 

4379.4 (M) 

Deviance 


53740 (S) 


53275 (S) 

51049 (S) 


51039 (S) 



54554 (M) 


53996(M) 

51734 (M) 


51724 (M) 

df 


3 


5 


7 


9 


Note: In each cell, the first value (top) is based on science scores (S), and the second value (bottom) is based on 
math scores (M). 

(*) The coefficient is not significant at the alpha level of .05 


In the bottom of Table 2, deviance values and degrees of freedom were reported for each level in each model. 
Deviance values can be used for comparing fitted-models. The difference between deviance values from two 
models and the difference between degrees of freedom from the same models can be used as a chi-square test 
(e.g. % 2 = Deviance mo deii - Deviance mo dei 2 , df = df 2 - dfi). Based on these comparisons, it was concluded that all 
models explained significantly more variance than the one-way random effects model which shows that the 
additional variables related to ICT were helpful to explain the achievement difference among students and 
schools. 

CONCLUSION 

The aim of this study was to explain students’ science and math achievement by looking at their use and 
accessibility of computers and related technologies, as suggested by Subrahmanyam et al. (2001). The results of 
this study indicated that students’ exposure to ICT at home and school was a strong predictor of their math and 
science performance. Students’ exposure to ICT out of school time had a larger impact on their math and science 
achievement than their exposure to ICT at school. This might point out the lack of the integration of ICT into 
classroom instruction at schools. Ziya et al. (2010) stated that students’ using computers in line with their needs, 
parents’ controlling the time their children use computers, the internet and computer for entertainment purposes 
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can be beneficial. The results of this study showed that ICT usage had a positive impact on students’ math and 
science performance in PISA. 

In this study, technology usage at school was found to be a weak predictor of math and science achievement. 
However, previous research showed that it may have still indirect impacts. Eskil et al. (2010), for example, 
indicated that some classroom activities have positive effects on students’ computer and technology use. Eskil et 
al. (2010) also argued that when students have advanced knowledge about computer technologies, they can be 
more successful in their studies. Therefore, direct and indirect effects of ICT usage at school should be taken 
into consideration. Also, Kubiatko and Vlckova (2010) found in their study that the amount of time spent using a 
computer had a positive and strong relation with science knowledge. The findings of this study support this idea. 
Students’ technology use may explain the science achievement gap. The same interpretation can be made for 
math achievement. Kim and Chang (2010) focused on math achievement gap between students coming from 
different racial and ethnic backgrounds. They found home computer use reduced the gap in math achievement. 

Unlike this study, Aypay (2010) found that there was no significant relationship between students’ use of ICT 
and academic achievement based on the results of PISA 2006. Aypay (2010) indicated that neither very frequent 
nor very little use of ICT improved student performance in PISA 2006. The 2005 curriculum reform in Turkey 
might be the main reason of this discrepancy. Turkey revised its curriculum and it has started using a 
constructivist approach since 2004 (Sahin, 2010). This reform required the integration of computers and other 
instructional technologies in classrooms. These changes in the curriculum might result in a positive relationship 
between ICT and student achievement in PISA 2009. 

The results of this study are limited to 15-years old students (9 th grades) in Turkey. Therefore, the results may 
not generalize to other age groups or other populations (e.g. students from other countries). 

Practical Implications of This Study 

Projects for comparing students’ achievement such as The Trends in International Mathematics and Science 
Study (TIMSS), Progress in International Reading Literacy Study (PIRLS), and The Programme for 
International Student Assessment (PISA) can enable countries to evaluate their system of education and to 
pursue their students in the fields of mathematics, science and reading by years rather than being projects for 
competition between countries (Ziya et al., 2010). This study focused on ICT usage and its effects on students’ 
achievement. The findings of this study can be beneficial for educators and policy-makers in education in terms 
of constructing classroom environments and designing curriculums. Aypay (2010) stated that Turkey first needs 
to lower the differences among schools. Turkey also needs to improve the use of ICT in educational system by 
adapting the technology in the content of the courses. Based on the results of this study, it seems that there is still 
a huge achievement gap between schools in Turkey. 

The results of this study can be also useful for comparing participating countries in PISA in terms of ICT usage 
and its effects on achievement. Previous international comparative studies showed that there are a number of 
factors influencing Turkish students’ performance in comparative examinations such as PISA and TIMMS. 
Ozgun-Koca & Sen (2002) found that very little use of computers, calculators and other instructional 
technology, intensive lecturing and note-taking in classrooms, loading students with too much information in the 
curriculum, and problems associated with measurement and evaluations were the main factors. Askar & Olkun 
(2005) found that the Turkish students’ access to computers in schools was quite low when compared to other 
OECD countries. The methodology of this study can be repeated using PISA results from other countries, and 
the results can be used for international comparisons. 
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